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The  problem  of  finding  a  point  with  largest  norm  in  a  bounded  polyhedral 
set  is  shown  to  have  a  considerable  range  of  complexity  depending  on  the  norm 
employed.  For  a  p-nornt  with  integer  p  2  1,  the  problem  is  shown  to  be  NP- 
complete.  For  the  “-norm,  the  problem  can  be  solved  in  polynomial  time. 

The  problem  of  finding  an  upper  bound  to  the  largest  norm  for  any  p  e  [1,°°] 
can  be  solved  in  polynomial  time  by  solving  a  single  linear  program. 
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SIGNIFICANCE  AND  EXPLANATION 


The  solution  set  to  many  important  constrained  optimization  problems  is  a 
set  that  is  bounded  by  planes.  When  such  a  set  is  bounded  it  is  useful  to 
find  the  size  of  a  largest  element  in  that  set.  In  this  work  we  show  that 
this  problem  may  be  extremely  easy  or  difficult  depending  on  the  measure  of 
size  (norm)  employed.  For  one  such  measure  the  problem  is  relatively  easy 
while  for  all  other  measures  it  is  intractable.  However  the  problem  of  merely 
finding  an  upper  bound  for  the  size  of  the  largest  element  turns  out  to  be  a 
surprisingly  simple  problem  that  can  be  solved  by  a  single  linear  program. 


A  VARIABLE-COMPLEXITY  NORM  MAXIMIZATION  PROBLEM 


O.  L.  Mangasarian  and  T.-H.  Shi.au 

1 .  Introduction 

The  problem  of  obtaining  bounds  for  polyhedral  sets  has  received  considerable 
attention  in  mathematical  programming  (14,  IS,  16,  12,  8,  9],  Part  of  the  significance  of 
this  problem  stems  from  the  fact  that  the  solution  set  to  a  linear  program  [4,  10]  and  to  a 
monotone  linear  complementarity  problem  [2]  is  such  a  polyhedral  set.  Bounding  the 
solution  set  to  such  problems  when  possible  is  then  of  practical  interest.  In  this  work  we 
shall  consider  the  polyhedral  set  X  in  Rn  defined  by 

(1.1)  X  (x  j  x  e  Rn,  Ax  >  b} 

where  A  is  a  given  m  x  n  real  matrix  and  b  is  a  given  m  x  1  real  vector.  He  assume 
throughout  this  work  that  X  is  bounded.  It  is  easy  to  show  that  a  necessary  and 
sufficient  condition  for  X  to  be  bounded  is  that 

(1.2)  Y  »  {y  I  y  e  Rn,  Ay  >  0,  y  M)  ■  ♦  . 

The  problem  we  wish  to  consider  here  is 

(1.3)  max  Ixl 

xex  p 

n  1/p 

where  1*1  denotes  the  p-norm  on  R*1,  1  &  p  <  “,  defined  by  Ixl  »  (  ^  l^r)  and 

"  p  i-1 

Ixl,  -  max  | x  |  • 

1  <i<n 

m  m 

We  will  show  that  while  (1.3)  can  be  solved  in  polynomial  time  for  p  «  “,  it  is  NP- 
complete  (6,  11]  for  integer  p  J  1.  Since  it  is  widely  believed  that  jio  NP-complete 
problem  can  be  solved  in  polynomial  time  (the  famous  conjecture  P  f  NP  in  computational 
complexity  theory),  the  difference  in  the  difficulty  between  p  *  »  and  all  other  integer 
p  i  1  is  enormous.  (The  standard  complexity  theory  terms  used  here  are  defined  in  Section 
4.)  In  fact  we  can  summarize  the  complexity  situation  for  our  problem  (1.3)  as  shown  in 
Table  1. 
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Problem 

Complexity 

Method  of  Solution 

1 .  Find  an  upper  bound 
to  max  Ixl  for 

xex  p 

any  p  e  (1,“] 

P 

(Deterministic 
polynomial  time) 

Single  linear  program 

2.  max  Ixl^, 
xex 

P 

2n  linear  programs 

3.  max  Ixl- 
xex 

NP-complete 
(Nondeterministic 
polynomial  time) 

2n  linear  programs 

4.  max  Ixl 
xex  P 

Integer  p  2  2 

NP-complete 

Vertex  enumeration 

Table  1. 


Complexity  of  max  Ixl  and  its  method 
of  solution.  xex  p 


We  note  in  passing  that  the  minimization  problem  min  Ixl  is  by  contrast  a  much 

xex  p 

simpler  convex  programming  problem  for  p  e  11,“].  In  fact  for  p  *=  1  and  “  it  can  be 
solved  by  standard  linear  programming  techniques  (4,  10]  or  by  a  polynomial  time  algorithm 
e.g.  [7].  For  p  *  2  the  problem  is  a  convex  quadratic  program  which  can  be  solved  by 
standard  techniques  e.g.  [2]  or  by  a  polynomial  time  algorithm  [3]. 

In  the  following  sections  of  this  paper  we  will  show  how  each  of  the  problem  of  Table 
1  is  solved  and  its  complexity.  Section  2  deals  with  finding  an  upper  bound  to  (1.3)  for 
p  e  [ 1 ,“] .  Section  3  deals  with  problem  (1.3)  for  p  «  1  and  “  while  Section  4  deals 
with  the  cases  of  integer  p  2  1. 


2. 


Bounding 


max  Ixl 

xex  P 


It  is  somewhat  surprising  that  for  any  p  e  [1,“],  an  upper  bound  to  the  solution  of 


the  nonconvex  problem 


max 

xex 


I  xl 


p 


can  be  obtained  by  solving  a  single  linear  program 


(Theorem  2.1  below).  This  is  especially  so  since  we  show  (Section  4)  that  the  problems 


max  Ixl  for  integer  p  2  1  are  intractable  NP-complete  problems.  When  X  is  contained 

xex  p 

in  the  nonnegative  orthant  r"  :=  {x  )  x  e  Rn,  x  >  0}  it  is  evident  that  a  solution  to  the 


1-norm  problem  max  Ixl.  is  easily  obtained  by  the  single  linear  progrrm 

xex  ! 


(2.1)  max  ex 

xexnR" 

where  e  is  a  vector  of  ones.  However  when  X  £  r",  ao  may  be  the  case  here,  solution  of 

max  Ixl.  will  take  2n  linear  programs,  as  shown  in  Section  3.  In  fact  we  will  show  in 

xex 

Section  4  that  the  problem  max  Ixl.  is  NP-complete.  However,  merely  obtaining  an  upper 

xex 

bound  to  max  Ixl  for  any  p  e  [1,“]  will  take  at  most  a  single  linear  program  as  shown 
xex  p 

by  the  following  result. 


2.1  Theorem.  Let  X  be  nonempty  and  bounded,  let 

(2.2)  B  :=  (A'TAr1AT  '  d!“  Bb 

and  let  B#^  denote  the  column  of  B.  Then  for  any  p  e  [I,**]  and  any  x  e  X 

(2.3)  Ixl  <  max  {Id!,  lYB. .  +  dl  } 

P  1<j<“  P  J  P 

where  Y  is  the  maximum  value  of  the  following  solvable  linear  program 

(2.4)  Y  :«  max{ey  |  x  6  r“,  y  6  R™,  Ax  -  y  «  b,  y  >  0}  . 

*,y 

Proof.  Note  first  that  the  boundedness  condition  (1.2)  implies  the  linear  independence  of 

the  columns  of  A  and  hence  the  nonsingularity  of  ATA.  In  addition  the  nonemptiness  and 

boundedness  of  X  implies  the  solvability  of  the  linear  program  (2.3).  Hence 

max  Ixl  =  maxdxl  |  x  e  Rn ,  y  e  Rm,  Ax  -  y  =  b,  y  >  0} 

P  P  * 

xeX  x,y  r 

=  maxdxl  |  x  -  By  +  d,  (AB-I)(y+b)  “  0,  y  >  0,  ey  <  y) 
x,y  p 

<  maxdxl  |  x  =  By  +  d,  y  >  0,  ey  <  y) 
x,y  P 

-  maxClBy  +  dl  |  y  >  0,  ey  <  y) 

y 

*  max  { I  dl  , 

1<j<m  P  J  P 


where  the  last  equality  follows  from  the  fact  that  the  maximum  of  a  convex  function  on  a 
bounded  polyhedral  set  is  attained  at  a  vertex  [13,  Corollary  32.3.4]. 

a 

Note  that  if  a  lower  bound  to  max  Ixl  is  also  desired,  then  we  have  the  following. 

xex  p 


2.2  Corollary.  Under  the  assumptions  of  Theorem  2.1  we  have  that 

I  By  +  dl  <  max  IxV 

p  =  xex  p 

where  y  is  a  solution  of  the  linear  program  (2.4). 

Since  by  Khachian's  result  [7]  a  linear  program  is  solvable  in  polynomial  time  in  the 
size  of  the  problem,  and  since  the  algebraic  operations  prescribed  in  (2.3)  can  all  be 
performed  in  polynomial  time,  the  following  holds. 


2.3  Corollary.  The  bound  (2.3)  can  be  computed  in  time  which  is  polynomial  in  the  size 
of  A  and  b. 

We  note  that  the  bound  (2.3)  of  Theorem  2.1  may  be  sharp  as  evidenced  by  the  following 
example. 


For  this  example  it  is  easy  to  verify  that 


max 

xex 

Ixl  = 

p 

10  for 

p  -  1,  2  and  "  ,  Y  =  42 

•  •  / 

-.0649 

.1688 

.0260^ 

|  ,  d  =  /  -1.0909\ 

.0519 

.0649 

-.2208  j 

!  [-  .7273/ 

Computing  the  bound  (2.3)  of  Theorem  2.1  gives  for  p  =  1,  2  and  <* 


It  is  rather  obvious  that  the  problem  max  Ixl^  can  be  solved  by  maximizing  the 

xex 

absolute  value  of  each  component  of  x  separately  subject  to  x  being  X.  This  leads  to 
the  following. 

3.1  Proposition 

The  problem  max  Ixl^  can  be  solved  by  solving  the  2n  linear  programs 

xex 

(3.1)  max  max{±x  |  x  e  Rn,  Ax  >  b) 

1  <i<n  1 

Since  each  linear  program  can  be  solved  in  polynomial  time  [7]  we  have  the  following. 


3.2  Corollary 


The  problem  max  Ixl^  can  be  solved  in  time  which  is  polynomial  in  the  size  of  A 

xex 

and  b. 

n 

Since  the  problem  max  Ixl.  is  equivalent  to  max  £  | x.  |,  its  solution  can  be 

xex  _ i=  1 


obtained  by  solving  2n  linear  programs  as  follows 


xex 


Proposition 

The  problem  max  Ixl.  can  be  solved  by  solving  the  2n  linear  programs 
xex 

(3.2)  max  max{vx  |  X  e  Rn,  Ax  >  b} 

vev  x 

where  V  is  the  set  of  2n  vertices  of  the  cube  in  Rn  defined  by 

(3.3)  {v  |  v  e  Rn,  -e  <  v  <  e)  . 

while  2n  linear  programs  can  be  solved  in  a  reasonable  amount  of  time  for  inter¬ 
mediate-sized  problems,  solving  2n  linear  programs  is  intractable  even  for  n  as  small 
as  15.  It  is  even  worse  for  general  p  e  (1,“)  if  we  try  to  enumerate  the  vertices  of 
X  for  finding  the  maximal  p-norm,  for  the  number  of  vertices  can  be  much  larger  than  2n. 
One  may  try  to  find  other  algorithms  that  are  computationally  effective.  Unfortunately,  as 
shown  in  the  next  section,  problem  (1.3)  with  p  f  •  is  no  easier  than  the  partition 
problem  (see  (4.1)  below)  which  is  inherently  intractable. 


We  begin  this  section  with  some  basic  concepts  of  complexity  theory  [6,  11).  Problem 
A  reduces  (in  polynomial  time'  to  problem  B,  denoted  by  A  “  B,  iff  the  following  holds: 

If  there  is  a  polynomial  time  algorithm  for  B,  then  one  can  construct  a  polynomial  time 
algorithm  for  A  using  the  algorithm  for  B  as  a  subroutine.  Problems  A  and  B  are  poly- 
nomially  equivalent  iff  A  “  B  and  B  «  A.  An  NP-complete  problems  is  one  which  is  poly- 
nomially  equivalent  to  any  one  of  the  standard  intractable  problems  such  as  the 
satisfiability,  partition,  or  travelling  salesman  problems  [6,  11).  These  problems  are 
considered  intractable  because  any  algorithm  which  solves  any  one  of  them  requires,  in  the 
worst  case,  an  amount  of  time  which  is  exponential  in  the  problem  size.  An  NP-hard  problem 
is  any  problem  such  that  some  NP-complete  problem  reduces  to  it  in  polynomial  time.  Thus 
an  NP-hard  problem  is  at  least  as  difficult  as  an  NP-complete  problem.  We  will  now  shown 
that  our  norm  maximization  problem  (1.3)  is  NP-hard  for  p  *  by  reducing  the  following 
NP-complete  partition  problem  to  it: 

(4.1)  Given  integers  c1 ,c2 , . . . ,cn ,  is  then  a  set  S  c  {l,2,...,n} 

such  that  I  c.  =  \  c.  ? 

jes  3  jf!S  3 

4.1  Theorem.  The  norm  maximization  problem  (1.3)  is  NP-hard  for  p6  [1,"). 

Proof .  We  will  show  this  by  reducing  (4.1)  to  (1.3).  Let  p  e  [1,“).  We  first  reduce 

(4.1)  to  the  following  problem: 

(4.2)  Given  integers  c^ ,C2 » • • • »cn,  is  there  an  x  e  jP  such  that: 

n 

1  ciXi  =  0,  -1  (  X;  1  1,  1  i  i  i  n,  lxl£  >  n  ? 

i=1 

It  .s  easy  to  see  that  (4.1)  has  a  solution  S  iff  (4.2)  has  a  solution  x  with  |x^|  «  1 
for  1  i  i  &  n  and  x^  =  1  for  i  e  S  and  x^  =  -1  for  i  ^  S.  Now  it  is  easy  to  see 
that  (4.2)  can  be  reduced  to  an  instance  of  problem  (1.3)  by  defining 


I 


and  answering  the  question: 

(4.3)  Is  max{lxl^  |  x  e  Rn,  Ax  >  b)  >  n^^  7 

Hence  if  we  can  solve  (1.3)  in  polynomial  tiir.a  we  can  solve  each  of  (4.3),  (4.2)  and  (4.1) 
in  polynomial  time.  Hence  (4.1)  «  (1.3)  and  (1.3)  is  NP-hard. 

D 

We  go  on  to  show  now  that  our  norm  maximization  problem  (1.3)  is  in  fact  NP-complete 
for  integer  p  /  In  order  to  do  this  we  introduce  additional  concepts  from  complexity 
theory.  A  nondetermini Stic  algorithm  is  an  algorithm  which  at  each  step  has  a  finite 
number  of  moves  from  which  to  choose  (instead  of  only  one  for  deterministic  algorithms)  and 
it  solves  a  problem  in  a  finite  sequence  of  choices  leading  to  a  correct  answer.  N£  is  the 
class  of  problems  solvable  by  a  nondeterministic  algorithm  in  polynomial  time,  including 
(4.1)  and  all  other  NP-complete  problems.  In  fact  NP-complete  problems  are  the  class  of 
most  difficult  problems  in  NP  in  the  sense  that  each  problem  in  NP  reduces  in 
polynomial  time  to  each  NP-complete  problem.  By  Cook's  theorem  [1,  6,  11],  all  we  need  to 
show  for  (1.3)  to  be  NP-complete  is  that  it  is  NP-hard  (which  we  already  have  done  in 
Theorem  4.1)  and  that  it  is  in  the  class  NP,  which  we  proceed  to  do  now.  In  order  to  do 
that  we  introduce  the  following  decision  problem  related  to  our  optimization  problem  (1.3): 

(4.4)  Given  A,  b  with  integer  entries  satisfying  (1.2),  and  nonzero  integers  r,  8,  p, 
is  there  a  vector  x  in  Rn  such  that 

Axib,  >xlp  >  ^  ? 

p  *  8 

Note  that  in  the  proof  of  Theorem  4.1  we  have  already  established  that  the  decision  problem 

(4.4)  in  NP-hard,  because  we  reduced  the  partition  problem  (4.1)  to  (4.2)  which  is  an 
instance  of  (4.4).  We  will  now  first  show  that  (4.4)  is  in  NP  and  hence  it  is  NP-complete. 
Then  we  will  show  that  an  optimization  problem  (1.3)  is  polynomially  equivalent  to  the  NP- 
complete  decision  problem  (4.4).  Note  that  condition  (1.2)  which  is  imposed  on  problem 


.4)  which  is  a  necessary  and  sufficient  condition  for  the  boundedness  of  X,  plays  an 
sential  role  in  Proposition  (4.2)  below  which  establishes  that  (4.4)  is  in  NP. 

2  Proposition.  Problem  (4.4)  is  in  NP  for  integer  p  2  1. 

oof .  It  follows  by  the  convexity  of  the  norm  and  the  boundedness  of  X  by  (1.2)  [13), 
pr  _  p  r 

at  Ixl*7  >  —  for  some  x  e  X  iff  »vl  >  —  for  some  vertex  v  of  X.  Moreover,  v 
p  =  s  P  =  s 

a  vertex  iff  there  is  a  J  c  {l,2,...,m},  |j|  =  n  such  that  v  is  the  unique  solution 
A^x  «  bi(  i  e  J,  and  AjX  2  b^  for  j  ^  J.  Consequently  we  can  prescribe  the 
llowing  nondeterministic  algorithm  for  solving  (4.4). 


3  Algorithm 

(i)  choose  J,  a  subset  of  {l,2,...,m}  with  cardinality  n. 

(ii)  Solve  A^x  =  b^,  i  e  J  for  one  x,  or  conclude  that  the  system  is 
inconsistent . 

(iii)  if  solution  x  found  and  A^x  2  b^  for  j  if  J  and  lxlp  >  —  then  print  x; 
-  -  ]  *  2  r  -  p  =  s  - 

success;  else  failure;  endif . 

ep  (ii)  can  be  performed  in  polynomial  time  (e.g.  by  Gaussian  elimination).  Since  we 
ve  assumed  that  p  is  an  integer,  ,xlp  can  be  evaluated  in  0( log2  p)  multiplications 
d  0( log2  n)  additions.  Hence  Algorithm  4.3  is  a  polynomial  time  algorithm  and  (4.4)  is 
NP. 


n 


Now  we  show  that  the  NP-complete  decision  problem  (4.4)  and  our  optimization  problem 
.3)  are  polynomially  equivalent,  thus  establishing  the  NP-completeness  of  (1.3).  First 
is  obvious  that  if  one  can  solve  the  optimization  problem  (1.3),  then  one  can  answer  the 
cision  problem  (4.4).  The  reverse  is  usually  done  by  a  binary  search  technique  showing 
at  the  optimization  problem  can  be  solved  by  a  polynomial  number  of  decision  problems, 
is  is  all  rather  obvious  for  discrete  combinatorial  problems,  but  not  for  our  continuous 


oblem  (1.3).  To  do  this  here,  we  shall  use  arguments  similar  to  those  of  Khachian  [7). 


tr.,n 

L  :=  l  log2(|A_|  +  1 )  +  }_  log2  v  |  |  +  1)  +  log2(nm+1)  +  log2(p+1) 

i,j-1  3  i 

L  is  the  total  length  of  binary  digits  representing  the  input  A,  b,  n,  m,  p  of  problem 
( 1.3) . 

4.4  Theorem.  For  any  integer  p  £  1,  problem  (1.3)  is  in  NP  and  hence  it  is  NP-complete. 

Proof .  Since  an  optimal  solution  of  (1.3)  is  at  a  vertex  of  X  (13),  such  a  vertex  can  be 

D,  D.  D  T 

r  12  n\ 

written  by  Cramer's  rule  as  (rj—  ,  -g-  -jj—J  <  where  D  and  are  determinants  of 


submatrices  of  [A  b) .  Hence 


D  T 


(i)  For  any  vertex  v  =  (~-  ,•••,  -jp]  ,  j D j  <  2^,  |d^|  <  2^,  lvlP  <  2*’*'.  (See  (5) 
for  details.) 

D1  D  T  B1  B  T 

(ii)  For  any  two  distinct  vertices  Ivl  ?  *wl  ,  v  =  (—  ,...,  — )  ,  w  =  (— 

p  p  U  D  DO 

it  follows  that 


iv»p  -  »wip! 


+  ...+  D 


B  ,  +  ...+  B_ 


P 


>  — - —  >  2-2pL 


!  B  1 1 


Hence  we  can  reduce  (1.3)  to  (4.4)  by  binary  search  on  the  interval  [0,2pL]  until  the 
range  is  less  than  2-2pL.  Since  each  iteration  reduces  range  by  half,  3pL  iterations 
will  do  that  if  we  have: 


(i)  t  *  0,  u  ♦  2pL 
( i i )  for  i  *  1  to  3pL  do 


if  answer  is  yes  then  i  else  u  *  -j  endif 


(iii)  solve  the  decision  problem  (4.4)  for  input  A,  b,  ~  =  -j  (£+u) 

(iv) 

( v)  end  for 

If  (iii)  can  be  done  in  polynomial  time,  then  (i)  to  (v)  can  be  done  in  polynomial  time. 
After  (v),  we  known  that  there  exists  an  x  e  X  such  that  i  =  u-2~^pL,  llxllp  >  t,  whereas 
there  is  no  x  e  X  such  that  *xlp  2  u.  Hnece  if  we  now  use  Algorithm  4.3  with  input 
•jj’  *  4 ,  A  and  b,  the  x  printed  in  step  (iii)  of  Algorithm  4.3  is  an  exact  vertex 
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solution  of  (1.3)  obtained  in  polynomial  time.  Hence  (1.3)  is  in  NP,  and  since  by  Theorem 


4.1,  (1.3)  is  NP-hard  it  follows  that  (1.3)  is  NP-complete. 


k 
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